SC-Track: a robust cell-tracking algorithm for generating accurate single-cell lineages from diverse cell segmentations

Abstract Computational analysis of fluorescent timelapse microscopy images at the single-cell level is a powerful approach to study cellular changes that dictate important cell fate decisions. Core to this approach is the need to generate reliable cell segmentations and classifications necessary for accurate quantitative analysis. Deep learning–based convolutional neural networks (CNNs) have emerged as a promising solution to these challenges. However, current CNNs are prone to produce noisy cell segmentations and classifications, which is a significant barrier to constructing accurate single-cell lineages. To address this, we developed a novel algorithm called Single Cell Track (SC-Track), which employs a hierarchical probabilistic cache cascade model based on biological observations of cell division and movement dynamics. Our results show that SC-Track performs better than a panel of publicly available cell trackers on a diverse set of cell segmentation types. This cell-tracking performance was achieved without any parameter adjustments, making SC-Track an excellent generalized algorithm that can maintain robust cell-tracking performance in varying cell segmentation qualities, cell morphological appearances and imaging conditions. Furthermore, SC-Track is equipped with a cell class correction function to improve the accuracy of cell classifications in multiclass cell segmentation time series. These features together make SC-Track a robust cell-tracking algorithm that works well with noisy cell instance segmentation and classification predictions from CNNs to generate accurate single-cell lineages and classifications.


INTRODUCTION
The analysis of time-resolved f luorescent microscopy images to obtain cellular dynamics at the single-cell level has enabled the detailed study of cellular events previously inaccessible to conventional cell biological approaches [1,2].This approach has led to the delineation of biological processes that induce a variety of cell fate decisions [3][4][5][6][7].Core to this analysis is the extensive use of f luorescent markers to mark single cells and to classify cellular states.However, generating single-cell tracks from these f luorescent timelapse microscopy images is often challenging, requiring laborious optimizations of f luorescent markers and imaging conditions [2].These optimizations are essential as good-quality cell segmentations are critical for accurate lineage tracing while avoiding biological artefacts caused by phototoxicity.
Deep learning-based convolutional neural networks (CNNs) are increasingly employed to overcome the inherent limitations of conventional f luorescence-based microscopy approaches [8].Among the most successful applications is the use of autoencoder CNNs, enabling computationally efficient image restoration of low-light microscopy images for deconvolution, denoising and generating super-resolution image reconstructions [9].Another area where CNNs have been successfully deployed is in the automated segmentation and classification of microscopy images [10][11][12][13].Recent work has shown that CNNs perform well in automatically detecting, segmenting and classifying heterogeneous cellular features of microscopy images, a task previously requiring time-consuming manual human annotations [13][14][15][16].
However, applying deep-learning CNNs in the automated segmentation and classification of f luorescent microscopy images presents another challenge for reliable cell tracking.State-of-theart CNNs are inherently noisy, with instances where objects fail to be detected or are misclassified [12,13,17,18].These inaccuracies pose a significant challenge for widely used cell-tracking solutions as these algorithms are not designed to confront the various object classification errors derived from CNNs.Thus, extensive finetuning of cell-tracking parameters and manual corrections of cell-tracking outputs are required, posing a significant barrier for many biologists as they often lack the technical know-how and time to undertake such tasks.
To address this challenge, we developed a robust generalized cell-tracking algorithm called Single Cell Track (SC-Track).SC-Track employs a hierarchical probabilistic cache-cascade model inspired by biological observations of cell division and movement dynamics of mammalian cells.We show that SC-Track can generate robust single-cell tracks from segmentation outputs from CNNs that contain missing segmentations to false detections.Furthermore, SC-Track functions well with whole cell or nuclear segmentations of diverse morphologies and imaging conditions without parameter optimizations.Finally, SC-Track can take noisy cell instance classifications and provide smoothed classification tracks to quantify cellular events accurately.

Tracking algorithm overview
SC-Track employs a tracking-by-detection approach whereby detected cells are linked between frames.A TrackTree data structure was used to store the tracking relationships between each segmented cell temporally and spatially (Figure 1A).Each branch of the TrackTree represents a single-cell lineage of the tracked instance of a segmented cell, where branch divisions indicate cell division events and the nodes on the branches represent the segmented instances of individual cells in a specific frame.The extracted features of the segmented cell are contained in each node of the TrackTree branch.
During the tracking process, SC-Track initializes the Track-Tree list with all cells from the initial frame, representing the initial single-cell tracks for the entire time-lapse sequence.To reduce computational costs, SC-Track will attempt to connect each segmented instance with its corresponding cell from the previous frame using a hierarchical tracking approach.SC-Track will initially examine the intersection over union (IoU) of the area between segmented cells between the current frame and preceding frame (Figure 2A).Segmented cells with only one overlapping segmentation are assumed to be high-confidence links and are assigned to the corresponding TrackTree.In situations with multiple segmented cells with overlapping IoUs, SC-Track will determine the correct cells by maximizing the similarity index between candidate cells between frames.When no segmented cell in the current frame overlaps with a segmented cell from the previous frame, SC-Track will expand the search area to identify possible candidates.
By recursively searching for candidate segmented cells from the previous frame, virtually all segmented cells can be accurately assigned to the correct TrackTree.If there are more segmented cells than the number of cached TrackTrees, three possible scenarios will be considered: (1) The orphan cell is a false detection.(2) The orphan cell is an actual detection of a cell recently migrated into the field of view.(3) A cell division event has occurred leading to the generation of daughter cells.

Detecting and assigning cell division events
In scenario three above, SC-Track will determine if a cell division occurred by searching for a potential mother cell in the mitotic state in the preceding frame (Figure 2B).When the segmented cell contains cell cycle classifications, SC-Track will allow cell division events at the TrackTree nodes where the mother cell is classified as in mitosis (M phase).However, a cell cycle-independent approach will be applied if no cell cycle information is available.In such cases, SC-Track will attempt to determine if a cell division event has occurred by matching an orphan segmented cell in the current frame to a potential mother cell in the previous frame using morphological data of the segmented cells.
To enable robust detection of cell division events without cell cycle data, SC-Track applies a series of rules based on well-established principles observed from mammalian cells undergoing cell division [19,20].When assigning a potential mother-daughter association from a possible cell division event, the following criteria must be met: (1) at least one unlinked segmented cell was found.(2) The segmented mother cell in the previous frame must be at least 1.3× the size of the segmented daughter cell in the following frame.(3) The candidate mother cell has not undergone a cell division event in the previous 50 frames (Supplementary Figure S1 shows the effect of varying this adjustable parameter).( 4) A candidate mother cell is identified in the expanded search area of the unlinked segmented cell.If a suitable candidate mother cell was identified in the previous frame for the orphan segmented cell, the TrackTree will be branched accordingly.However, if no appropriate candidate mother cells are found, SC-Track will assume a new detection event may have occurred.The following section will discuss the algorithm employed to address the possibility of new detection events and the initialization of new TrackTrees.

Cache matching frames to address false and missing detection events
The stochastic nature of CNNs may lead to instances where some cells fail to be detected or are false detections [12,13,17,18], resulting in TrackTrees that contain inaccurate cell segmentations.We developed a cache-matching algorithm to address the stochastic loss of detected instances in the segmented cells (Figure 3A).In this process, SC-Track will follow a tiered response based on the likelihood that the instance segmentations are accurate.Initially, SC-Track will focus on generating highconfidence TrackTrees with cell segmentations that appear consecutively across multiple frames.When cell segmentations cannot be linked to an initialized TrackTree from the previous frame, and no suitable candidate mother cell is identified, SC-Track will search for up to five preceding frames for potential orphan TrackTrees to link (Figure 3A).If an unlinked TrackTree is found within five frames, SC-Track will assign the matched segmented cell to the corresponding TrackTree.When no matches are found, SC-Track will assume a new detection event may have occurred, and a new TrackTree will be initialized.
After all segmented frames are analysed, SC-Track will review all TrackTrees to filter high-confidence TrackTree initializations (Figure 3B).SC-Track assumes that TrackTrees that contain only one node are false-positive cell segmentations since no linking segmentations can be detected in subsequent frames.Therefore, these one node TrackTrees will be excluded in the linking process.For TrackTrees containing at least two nodes, SC-Track will assume that some of these TrackTrees are incomplete TrackTree fragments.To identify and repair these incomplete TrackTree fragments, SC-Track will survey the terminal ends of all TrackTrees to determine instances where a terminal end of one TrackTree is spatially and temporally proximal to a terminal end of another Track-Tree.When such instances are detected, SC-Track will attempt to pair the first instance of an unlinked TrackTree with the last instance of another unlinked TrackTree.The linking of TrackTrees will follow two requirements: (1) the last segmented cell of the preceding unlinked TrackTree is within the expanded search area of the first segmented cell of the unlinked TrackTree.(2) The last segmented cell of the preceding TrackTree is within three subsequent frames of the first segmented cell of the unlinked TrackTree.If more than one candidate TrackTree is available for linkage, the similarity index of the last segmented cell instance of both competing candidate TrackTrees will be calculated.The TrackTrees containing the segmented cell instance with the highest similarity index will be linked ( Figure 3B).After the linking of terminal ends of track trees is completed, short TrackTree initializations that span less than 10 frames are discarded to remove false detection instances.Finally, the intervening gaps between the linked TrackTrees will be filled with the preceding cached cell segmentation in the linked TrackTree.

Instance classification smoothing
Instance classification of multiclass cell segmentations is often noisy [21].We have implemented a class smoothing function to smooth out noisy classification of cells that transition from one cellular state to another.We developed the TrackTree Class Smoothing (TCS) algorithm to automatically correct the predicted results of cell type classifications (Figure 3C).TCS assumes that a cell classification change is more likely to occur in a time series Figure 2. Schematic illustration summarizing the hierarchical tracking approach for single-cell-tracking and identifying cell division events.(A) SC-Track employs a hierarchical cell-tracking approach to minimize computational costs.The overlap between the segmented cells of the preceding and subsequent frame determines the linking of segmented cells between frames.If only one cell segmentation overlaps, the segmented cell in the subsequent frame is automatically linked to the respective TrackTree in the preceding frame.When there are multiple overlapping cell segmentations, the identification of the linked cells will be determined by the similarity value of the overlapping cell segmentations.If no overlapping candidate segmented cell was identified, the bounding box of the unlinked segmented cell will be expended to identify possible candidates.(B) When a segmented cell instance that cannot be linked to available TrackTrees is identified, SC-Track will attempt to determine if a cell division event has occurred.SC-Track will perform a search in the preceding frame to determine if a compatible candidate mother cell is available.When a compatible mother cell is detected, the segmented cell instance will be branched to the corresponding TrackTree, and a cell division event is recorded.If no compatible mother cell is identified, SC-Track will assume that the segmented cell is due to a recent appearance of a cell in the microscope field of view and a new TrackTree is initialized.When a segmented instance of a cell is detected but no compatible TrackTree is available in the previous frame, SC-Track will perform a cached search of up to five preceding frames to identify a compatible unlinked TrackTree.If a compatible TrackTree is found, the segmented instance will be linked to the TrackTree.Missing gaps in the TrackTree will be filled with the last cached cell segmentation in the preceding frame.(B) Diagram describing the TrackTree linking process.When all segmented cells are analysed, SC-Track will attempt to link TrackTrees that contain more than a single node.The linking process is based on the last segmentation of the TrackTree and the first segmentation of a separate TrackTree.When more than one compatible TrackTree is available for linkage, the cell segmentation with the highest similarity will be linked.Missing gaps in the linked TrackTrees are filled with the last cached cell segmentation in the preceding frame.(C) When a multi-class cell segmentation is performed, it is often observed that erroneous cell classifications would occur stochastically.The TCS algorithm employs a probabilistic cached search algorithm to determine if a class switch has occurred for the respective segmented cell in a time series.Note: The vertical dotted line represents a frame of a timelapse image series, and the circles represent an instance of a segmented cell.An established track tree is represented by a horizontal line running through the circles across the frames.
when the same cell maintains the same cell classification over several frames.To evaluate the accuracy of the cell class change, TCS adopts a probabilistic cached search model.This search process is confined to the individual branch of the TrackTree and does not extend beyond the cell division branch.
The TCS probabilistic cached search model functions with the following logic: During the initialization of the TrackTree, TCS will automatically adopt the initial classification of the detected cell instance as the default class.When TCS detects an instance where the tracked cell undergoes a cell classification change to Type A, the algorithm will undertake a forward search on the TrackTree to count the number of Type A classifications in the subsequent nine frames.When the number of nodes classified as Type A exceeds a probability threshold of 60%, TCS will conclude that a change in cell classification has occurred and will update the default classification as Type A. Otherwise, the node where Type A was first detected will be corrected to the default cell classification.Exiting the default Type A classification occurs when a Type B class change is detected.To determine if a class change has occurred, TCS performs a forward TrackTree search in the subsequent nine frames for the Type B classification.When the number of nodes of Type B classifications exceeds the probability threshold of 60%, the new default Type B classification is adopted.This process can be repeated to multiple cell classifications.

Calculating similarity index when connecting segmented cells between frames
When there is more than one segmented cell overlapping with the previous frame, SC-Track will select the segmented cell with the highest similarity value with the segmented cell in the previous frame.SC-Track will employ the following formula to determine the similarity value of the possible candidate pairs between frames: S i = {P 1 , P 2 , . . .P n } , where P n = x n , y n sm i,j = IoU S i , S j + Dis S i , S j + Sps S i , S j + Sas S i , S j + Δ S i , S j S i represents the set of contour points in a 2D space defined by x n , y n for points P 1→n of a cell.sm i,j represents the similarity index between the segmented cell i in the previous frame and the segmented cell j in the subsequent frame.Dis is the calculated distance between the centroid of the segmented cell i in the previous frame and the centroid of the segmented cell j in the current frame.IoU represents the intersection over union of the contours of cells i and j.Spsrepresents the shape similarity value [22], and Sas represents the area similarity of the two cells.Δ S i , S j represents additional supplementary features, such as the similarity in the variance or total intensity of f luorescent signals from segmented cells.To calculate IoU S i , S j , Dis S i , S j , Sps S i , S j , Sas S i , S j and Δ S i , S j , the following formula was employed: where where mean (s i ) < mean s j and var (s i ) < var s j .

Bounding box search area expansion
When no segmented cell in the preceding frame is detected in the segmented area of a cell in the current frame, SC-Track will expand its search area to search for potential candidates.The expansion of the search area utilizes the bounding box of the segmented cell, which is expanded with the following formula: Bc represents the bounding box of a cell.Pos represents the position of the bounding box with the minimum value of the segmented cell in the x-axis and y-axis represented by x 1 and y 1 while the maximum value as x 2 and y 2 , respectively.Ecrepresents the expanded bounding box where potential cell candidates located in the current frame can be matched to the previous frame; α represents the coefficient for expanding the bounding box.By default, α is set to 1.5 (Supplementary Figure S2 shows the effect of varying this adjustable parameter).

Evaluation of cell-tracking performance using MOTA and IDF1
To evaluate the performance of cell trackers in accurately tracking segmented cells, we used performance measures established in the Multiple Object Tracking (MOT) framework, which includes IDF1 [23] and MOTA [24,25].IDF1 measures how long a tracker accurately assigns segmented cells to the correct single-cell lineage over time.It represents the ratio of correctly identified detections over the average ground-truth and computed detections [23].IDF1 is calculated from the following formula: , where IDP = IDTP IDTP + IDFP and IDR = IDTP IDTP + IDFN IDP represents the cell-tracking identification precision.It is computed as the average ratio of accurately identified true positives divided by the sum of accurately identified true positives and inaccurately classified false positives.IDR represents the identification recall computed as the average ratio of accurately identified true positives divided by the sum of accurately identified true positives and failed detections of each single cell track.
The multiple objects tracking accuracy (MOTA) measures the overall accuracy of the tracker performance by measuring how often a mismatch occurs between the tracking results and the ground truth [24,25].The MOTA score is determined from the total number of errors for false positives (FP), missed targets (FN) and identity switches (IDsw) normalized over the total number of ground-truth (GT) tracks.This measure is computed using the following formula: The cell-tracking outputs used to calculate the IDF1 and MOTA values are available from Zenodo: https://zenodo.org/doi/10.5281/zenodo.8284986.The Python scripts used to analyse the celltracking results can be found on GitHub: https://github.com/chan-labsite/SC-Track-evaluation.

Evaluating cell division detection events using the CDF1 score
Generating single-cell lineages over multiple cell division events requires the reliable detection of cell division events and the accurate assignment of mother-daughter relationships.We established the Cell Division F1 score (CDF1) as IDF1 and MOTA do not measure cell division events reliably.This metric is based on the principles of the widely used F1 score and is defined by the formula: CDTP indicates an actual positive cell division event, where both daughter cells of a cell division event are accurately identified and assigned to the correct TrackTree.CDFP indicates a false-positive cell division event, where daughter cells are incorrectly assigned to a TrackTree and classified as a cell division event.CDFN indicates a false-negative cell division event, where a cell division event occurred but is not detected or the mother-daughter cells were inaccurately assigned to the wrong TrackTree.The cell-tracking outputs used to benchmark the CDF1 results are available from Zenodo: https://zenodo.org/doi/10.5281/zenodo.8284986.The Python scripts used to analyse the cell-tracking results can be found on GitHub: https://github.com/chan-labsite/SC-Track-evaluation.
The timelapse microscopy images used to develop SC-Track were generated from cells cultured under the conditions described above.The images were saved as individual multichannel TIFF files.Four time-lapse movies with varying cell densities were generated (Supplementary Table S1).These datasets were automatically segmented using a custom pretrained StarDist model [12] and manually corrected using the VGG Image Annotator (VIA) [26] to remove false and inaccurate classifications.The annotated files contain the cell contour and 'cell cycle phase' class information.The contour information was converted into a mask with values ranging from 1 to 255.The uncorrected and corrected mask images, along with the original mCherry channel image, constitute the datasets used to finetune the tracking parameters of SC-Track.
For the testing datasets, three RPE1 microscopy timelapse image series and two MCF10A microscopy timelapse image series were automatically segmented using our custom-trained StarDist model and manually corrected to ensure accuracy of the instance segmentations, cell classifications and identity of single-cell lineages (Supplementary Table S2).The imaging conditions were described above with a sampling frequency of 5 min.To test the reliability of SC-Track to track segmented cells with missing or false-positive instances accurately, we utilized the uncorrected segmentations of the testing dataset (Supplementary Table S3).In addition, to assess how SC-Track can cope with varying levels of missing cell segmentations, we randomly deleted additional segmented cells from each frame to varying degrees to simulate higher levels of missed segmentations (Supplementary Table S4).The in-house generated segmentation masks, custom-trained StarDist model and ground truth-tracking results used in testing SC-Track can be obtained from Zenodo: https://zenodo.org/doi/10.5281/zenodo.8284986.

Cell-tracking challenge dataset
We used the silver reference segmentation results from the Cell Tracking Challenge (CTC) to test the SC-Track performance with various cell types and segmentation modes [27].The silver reference datasets are derived from the uncorrected segmentation results obtained from various custom-trained CNNs applied on mammalian cell lines with different morphological appearances and imaging conditions (Supplementary Table S5).The silver reference masks and ground truth-tracking results were obtained from the CTC website (http://celltrackingchallenge.net/2d-datasets/).

Generation of single-cell lineages from segmentation masks
The segmentation results from the various evaluation datasets were used to measure the cell-tracking performance of SC-Track and three other trackers: pcnaDeep [28], Deepcell-tracking [29] and TrackMate [30,31].For cell-tracking experiments involving in-house generated testing datasets, the segmentation results in the form of a VGG image annotator (VIA2) compatible JSON file containing cell cycle class information of each segmented cell was used [26].The data in the JSON files were read directly by SC-Track and pcnaDeep to generate the cell lineage tables.The cell segmentation data in the JSON files were converted into greyscale multi-TIFF image files before being read by TrackMate and  1.We could not perform the cell-tracking experiments with the CTC dataset on pcnaDeep as it requires cell cycle class information to function [28].The scripts used to generate the cell-tracking results are available from GitHub: https://github.com/chan-labsite/SC-Track-evaluation.

Evaluating automated cell cycle class correction
To evaluate the cell cycle class accuracy from the TCS function of SC-Track, we computed the F1 score of uncorrected raw cell classifications obtained from our custom pre-trained StarDist model and compared it with the cell cycle-corrected F1 score of SC-Track (Supplementary Table S3).The F1 score is calculated with the following formula:

Runtime and multi-platform compatibility testing
We conducted runtime efficiency tests on Windows, Linux and macOS platforms.All tests were performed using the same dataset and repeated three times.The Windows platform was equipped with an AMD R7 3700X CPU, RTX 2080 GPU and 16GB of RAM.The Linux platform was equipped with an Intel i7 11800H CPU, RTX 3050Ti GPU and 16GB of RAM.The macOS platform is a 2021 MacBook Pro with an M1 processor and 8GB of RAM.

Evaluation of SC-Track cell-tracking performance
The overall performance of SC-Track in cell tracking was measured using two metrics: the MOTA [24,25] and the harmonic mean of Identification Precision and Recall (IDF1) [23].We employed two cell-tracking metrics because IDF1 is more sensitive to the total duration of incorrect track assignments, while MOTA is more sensitive to the total number of track switches [23].We also introduced a new metric, the Cell Division F1 score (CDF1), to measure the cell tracker's ability to reliably detect cell division events and accurately assign mother-daughter cell relationships.The CDF1 score is essential for measuring single-cell lineage reconstructions over multiple cell division events, which the IDF1 and MOTA scores cannot reliably capture.For comparison, we benchmarked SC-Track against three other freely available celltracking algorithms that provide similar functionalities ( Table 1): TrackMate [30,31], Deepcell-tracking [29] and pcnaDeep [28].
The initial tests focused on generating single-cell tracks from nuclear masks obtained under ideal conditions, using manually corrected nuclear segmentation masks with accompanying cell cycle classifications with a 5-min temporal resolution (Table 2).
The results show that with ideal segmentation results, SC-Track performed best in all three measured metrics, obtaining an average value of 0.992 for IDF1, 0.998 for MOTA and 0.952 for CDF1.These results were comparatively higher than the second-bestperforming cell-tracking algorithm pcnaDeep, which obtained an average value of 0.967 for IDF1, 0.984 for MOTA and 0.526 for CDF1, respectively.
To further test SC-Track in generating accurate single-cell lineages, we resampled our original test dataset to mimic imaging time intervals of 10, 15 and 20 min.The increase in time intervals poses a more challenging cell-tracking problem as each cell has more time to migrate and change morphologically between successive frames.Our results show that SC-Track gives the best overall IDF1, MOTA and CDF1 scores in the 5-min interval, but as expected, its performance is reduced at longer intervals (Figure 4).Despite the reduced performance, SC-Track maintained the best overall performance in all three metrics across all time intervals tested.These results indicate that SC-Track works well with different temporal resolutions (Figure 4).
CNN instance segmentations generally display errors ranging from missing to inaccurate segmentations [12,13,17,18].To assess the various tracker's ability to generate reliable singlecell lineages from noisy CNN-based cell segmentations, we repeated the tests with uncorrected image segmentations from our custom-trained StarDist models (Supplementary Table S3).Our results show decreased tracking accuracy for all tested trackers, while SC-Track again gave the best overall performance, maintaining the highest average MOTA, IDF1 and CDF1 scores throughout, obtaining an average value of 0.955 for IDF1, 0.988 for MOTA and 0.889 for CDF1 (Table 3).The results were comparatively higher than the second-best tracker, pcnaDeep, which obtained an average value of 0.944 for IDF1, 0.973 for MOTA and 0.362 for CDF1, respectively.To further examine SC-Track's ability to overcome missing instances of cell segmentations, we generated a synthetic test dataset where cell instances were randomly removed at varying degrees (Supplementary Table S4).Our results show that SC-Track's cache matching algorithm can compensate for the loss of instance detections while maintaining a high IDF1 and MOTA score throughout all tested conditions (Figure 5A and B).Furthermore, despite increasing missing instance detections, SC-Track remains reliable in detecting cell division events (Figure 5C).
To examine the generalizability of SC-Track, we used several publicly available microscopy datasets containing diverse cell segmentations extracted from different cell types and imaging conditions (Supplementary Table S5).We decided to use the silver reference segmentation results from the CTC, as they were derived from the segmentation results of CNN models [27].These cell segmentations are based on a collection of timelapse microscopy images taken with various imaging settings, cell morphologies and a mixture of nuclear to whole-cell segmentations [27].Furthermore, the silver reference segmentations are accompanied by ground truth-tracking results, making these datasets an impartial real-life test for SC-Track.Our results show that SC-Track consistently displayed the best cell-tracking performance measured by MOTA and IDF1 scores for all the CTC datasets tested, obtaining an average IDF1 score of 0.988 and an average MOTA score of 0.997 (Table 4).These scores were significantly higher than the results obtained from TrackMate and Deepcell-tracking, which obtained average scores of 0.665 and 0.689 for IDF1 and 0.664 and 0.627 for MOTA, respectively.Using only the silver reference segmentation results, SC-Track can reliably detect cell division events on most CTC datasets, obtaining an average score of 0.955 (Table 4).These results suggest that SC-Track is a robust generalized cell-tracking algorithm that performs equally well on various cell segmentation types and can be used under challenging conditions where the cell segmentation dataset exhibits high levels of detection loss.

Instance classification smoothing of single-cell tracks and runtime evaluations
It is well established that CNNs occasionally misclassify multiclass instance segmentations.Instance misclassifications are caused by objects that partially fit into a particular class or suboptimal imaging conditions.The inherent noise in the cell classifications can pose a problem if accurate classifications of cellular states are essential, such as in the quantification of cell cycle phases in an image time series [28].To overcome this problem in SC-Track, we developed a TCS algorithm that employs a probabilistic cached class smoothing approach to identify cell phase transition points accurately.To evaluate the utility of TCS, we measured the F1 scores of our custom-trained StarDist model to classify the cells in our test dataset using the f luorescent PCNA cell cycle reporter signal (Table 5).The results indicate that TCS can improve the F1 classification scores of all cell cycle classes.Finally, we conducted runtime tests for SC-Track to determine how long SC-Track takes to generate single-cell tracks from cell segmentations.We measured the time taken to analyse cell segmentations from microscopy timelapse series of varying lengths (50-500 frames) and compared it with TrackMate, Deepcell-tracking and pcnaDeep.Our results show that when working with small to moderate imaging datasets (50 frames to 200 frames), SC-Track had the best performance ( Figure 6).However, we noticed that increasing the number of analysed frames decreases the processing speed of SC-Track (Figure 6).This was mainly caused by the time spent loading the timelapse microscopy images before generating the single-cell lineages by SC-Track.In contrast, TrackMate processing speed increased with larger imaging datasets, exhibiting a superior performance compared to SC-Track when processing imaging datasets containing more than 200 frames.

DISCUSSION
Generating accurate single-cell lineages from timelapse microscopy images is critical to obtaining reliable cellular dynamics for quantitative analysis.Due to the immense diversity in cell morphology, movement dynamics and imaging conditions,  We were unable to evaluate pcnaDeep's cell-tracking performance on the CTC dataset because pcnaDeep requires cell cycle data encoded in the cell segmentations to generate single cell tracks.The results for Deepcell-tracking CDF1 scores were not included as the tracker failed to detect any cell division instances in all the datasets tested.Only timelapse movies containing more than 10 cell division events are included in the analysis of CDF1 scores to ensure a reliable measure of cell division-tracking performance.
it is assumed that no single cell-tracking algorithm could perform universally well.This necessitates testing several tracking algorithms and extensive parameter tuning before obtaining an optimal cell-tracking solution.One major challenge in developing a generalizable cell-tracking algorithm is the inherent fixed assumptions hard-coded into the computational algorithm that links segmented cells.When these assumptions are incorrect, the linking process could fail, leading to the generation of incorrect single-cell lineages.
A popular method used in cell tracking is a multi-stage approach where different algorithms are applied to different aspects of the cell-tracking problem [32].SC-Track's hierarchical probabilistic cache-cascade model is inspired by this multistage approach and is designed to mimic how humans identify  linked cells between frames.SC-Track prioritizes linking cells between frames that are most likely correct first and removes these linked cells from consideration before applying subsequent linking methods.By employing this approach, the computational problem of linking more challenging cells (for example, cells that exhibited significant morphological changes between frames) can be simplified, making it easier to link these cells between frames accurately.To improve SC-Track's generalizability, the cell dimensions from segmented cells were used to decide suitable distance parameters.For example, by setting the expanded search area relative to the size of the segmented cell, SC-Track bypasses the requirement for the user to decide on an appropriate distance search parameter.This approach, combined with the primary overlap-based cell-tracking method, makes SC-Track insensitive to cell morphology and size changes caused by different cell types or objective magnifications used during imaging.Consistent with the view, we find SC-Track to perform equally well with all CTC datasets, which contain various cell types, nuclear or whole cell segmentations, imaged with various objective lenses (ranging from 4× to 63×), with variable image pixel sizes (varying from 512 × 512 to 2048 × 2948) and variable temporal resolutions (from 5 to 29 min) ( Table 4, Supplementary Table S5).
In contrast, TrackMate and Deepcell-tracking produced mixed results, producing relatively accurate tracks in a subset of the CTC datasets, indicating a need for parameter adjustments to improve tracking performance (Table 4).Furthermore, SC-Track's hierarchical probabilistic cachecascade model tolerates false or missing cell segmentations caused by the stochastic nature of CNNs, reducing the need for extensive and time-consuming manual corrections of image segmentations.We further implemented a cache smoothing algorithm to reduce the stochastic noise in cell classifications from CNNs while increasing the accuracy of the cell classifications in time series.These functionalities are computationally efficient, allowing SC-Track to be run locally without requiring access to high-performance computing clusters.
Despite SC-Track advantages in cell tracking, there are some limitations.Firstly, SC-Track experiences a reduction in computational efficiency as the number of frames increases.While SC-Track's primary overlap-based cell-tracking method is computationally efficient, the cache searching, gap filling and TrackTree linking functions (essential features to overcome false and missing cell segmentations from CNN-based segmentation results) are performed after the initial cell-tracking process.This necessitates the storage of all TrackTrees in memory, and the computational cost of performing these post-tracking functions decreases computational efficiency and is proportional to the number of frames in the imaging dataset.We do not imagine this limitation to be a barrier to most users since our Windows test system, equipped with an AMD R7 3700X CPU, RTX 2080 GPU and 16GB of RAM, regularly takes less than 10 min to process timelapse microscopy images with 1000 frames.Secondly, SC-Track currently supports only 2D image analysis.No immediate plans exist to upgrade SC-Track to include support for 3D image analysis.However, we imagine the cell-tracking approach employed by SC-Track can be applied effectively in 3D space with additional computational overheads due to incorporating an additional dimension in the linking cells in a TrackTree.Thirdly, the SC-Track class smoothing algorithm makes generalized assumptions that the degree of cell classification errors is equal across all classes and that the frequency for each cell class appearing in a timelapse series is similar.These assumptions, while reasonable, may not be valid under all biological conditions.From our observations, optimal TCS results can be achieved by determining the likelihood of classification errors for each cell class before applying an appropriate class smoothing weight to each class separately.
In summary, SC-Track solves a longstanding problem involving the use of CNNs in automated segmentation and classification of cells from time-lapse microscopy images.By encoding biological intuition in SC-Track's cell-tracking SC-Track performs well in diverse segmentation without finetuning parameters.A Python implementation of SC-Track is available,

Figure 1 .
Figure 1.Schematic illustration providing an overview of SC-Track, the TrackTree data structure and analysis pipeline.(A) Summary of the TrackTree data structure.Each linked segmented cell is tracked in a TrackTree.A node in a TrackTree branch represents an instance of the segmented cell in a particular frame with its accompanying cell segmentation information.A branching of a TrackTree represents a cell division event.(B) Simplified overview of the analysis pipeline of SC-Track.Instance segmentation of cells from each frame is sequentially added to their respective TrackTrees.The hierarchical probabilistic cache-cascade model of SC-Track determines the assignment of each instance segmentation.If cell classification information is encoded in the TrackTrees, SC-Track will employ the TCS algorithm to correct the noisy cell classifications.

Figure 3 .
Figure 3. Schematic illustration describing SC-Track cached matching algorithm and TCS.(A) Diagram describing the cache matching algorithm.When a segmented instance of a cell is detected but no compatible TrackTree is available in the previous frame, SC-Track will perform a cached search of up to five preceding frames to identify a compatible unlinked TrackTree.If a compatible TrackTree is found, the segmented instance will be linked to the TrackTree.Missing gaps in the TrackTree will be filled with the last cached cell segmentation in the preceding frame.(B) Diagram describing the TrackTree linking process.When all segmented cells are analysed, SC-Track will attempt to link TrackTrees that contain more than a single node.The linking process is based on the last segmentation of the TrackTree and the first segmentation of a separate TrackTree.When more than one compatible TrackTree is available for linkage, the cell segmentation with the highest similarity will be linked.Missing gaps in the linked TrackTrees are filled with the last cached cell segmentation in the preceding frame.(C) When a multi-class cell segmentation is performed, it is often observed that erroneous cell classifications would occur stochastically.The TCS algorithm employs a probabilistic cached search algorithm to determine if a class switch has occurred for the respective segmented cell in a time series.Note: The vertical dotted line represents a frame of a timelapse image series, and the circles represent an instance of a segmented cell.An established track tree is represented by a horizontal line running through the circles across the frames.
Recall where Precision = TP TP + FP and Recall = TP TP + FN The ground-truth cell cycle classification used to compute the F1 score was obtained by manually correcting the automated cell cycle classifications from our custom StarDist model.The JSON file containing the raw uncorrected cell segmentations and the cell cycle classification data used to compute the F1 results is available on Zenodo: https://zenodo.org/doi/10.5281/zenodo.8284986.The scripts used to compute the F1 scores of individual cell cycle phases are available on GitHub: https://github.com/chan-labsite/SC-Track-evaluation.

Figure 4 .
Figure 4. Evaluation metrics of the cell-tracking accuracy based on ground truth segmentations.Box plots of (A) IDF1, (B) MOTA and (C) CDF1 scores for all four cell trackers in varying imaging time intervals.Each point displayed on the boxplots represents the respective scores of the five test datasets.The line in the boxplot represents the median.The results for Deepcell-tracking CDF1 scores were not included in (C) as the tracker failed to detect any cell division instances in all the datasets tested.

Figure 5 .
Figure 5. Evaluation metrics of the cell-tracking accuracy based on varying cell segmentation loss.Box plots of (A) IDF1, (B) MOTA and (C) CDF1 scores for all four cell trackers challenged with the ground truth segmentation dataset with randomly deleted segmentations to mimic varying levels of cell segmentation loss.Each point displayed on the boxplots represents the respective scores of the five test datasets.The line in the boxplot represents the median.The results for Deepcell-tracking CDF1 scores were not included in (C) as the tracker failed to detect any cell division instances in all the datasets tested.

Figure 6 .
Figure6.Runtime evaluation comparisons.The number of frames each tracker can process in one second is displayed on the y-axis, while the xaxis represents the varying number of processed image frames.The solid line represents the average performance, with the shaded area representing the 95% confidence interval for each cell tracker on three different computer systems running Windows, Linux or macOS, respectively.

Table 2 :
IDF1, MOTA and CDF1 test results based on ground truth segmentation masks The best scores for each respective dataset and the best average score are highlighted in bold.The results for Deepcell-tracking CDF1 scores were not included as the tracker failed to detect any cell division instances in all the datasets tested. Note:

Table 3 :
IDF1, MOTA and CDF1 test results based on uncorrected segmentation masks The best scores for each respective dataset and the best average score are highlighted in bold.The results for Deepcell-tracking CDF1 scores were not included as the tracker failed to detect any cell division instances in all the datasets tested. Note:

Table 4 :
CTC imaging dataset test results for SC-Track, TrackMate and Deepcell-tracking Note:

Table 5 :
F1 cell cycle classification test results obtained from raw StarDist cell classification predictions compared with TCS-corrected cell classifications TCS-corrected F1 values that show an improvement over the Raw F1 values are highlighted in bold in each category. Note: